x86/smp: use APIC ALLBUT destination shorthand when possible
If the IPI destination mask matches the mask of online CPUs use the
APIC ALLBUT destination shorthand in order to send an IPI to all CPUs
on the system except the current one. This can only be safely used
when no CPU hotplug or unplug operations are taking place, no
offline CPUs or those have been onlined and parked, all CPUs in the
system have been accounted for (ie: the number of CPUs doesn't exceed
NR_CPUS and APIC IDs are below MAX_APICS) and there's no possibility
of CPU hotplug (ie: no disabled CPUs have been reported by the
firmware tables).
This is specially beneficial when using the PV shim, since using the
shorthand avoids performing an APIC register write (or multiple ones
if using xAPIC mode) for each destination when doing a global TLB
flush.
The lock time of flush_lock on a 32 vCPU guest using the shim in
x2APIC mode without the shorthand is:
Global lock flush_lock: addr=
ffff82d0804b21c0, lockval=
f602f602, not locked
lock:
228455938(
79406065573135), block:
205908580(
556416605761539)
Average lock time: 347577ns
While the same guest using the shorthand:
Global lock flush_lock: addr=
ffff82d0804b41c0, lockval=
d9c4d9bc, cpu=12
lock:
1890775(
416719148054), block:
1663958(
2500161282949)
Average lock time: 220395ns
Approximately a 1/3 improvement in the lock time.
Note that this requires locking the CPU maps (get_cpu_maps) which uses
a trylock. This is currently safe as all users of cpu_add_remove_lock
do a trylock, but will need reevaluating if non-trylock users appear.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>